Method and system for generating an audio or midi output file using a harmonic chord map

ABSTRACT

Techniques are provided for generating an output file. One technique involves the steps of generating audio or MIDI content blocks from one or more musical performances; receiving an input file having audio or MIDI music content; generating a harmonic chord map for the input file; using the harmonic chord map to automatically select a subset of the audio or MIDI content blocks, and generating the output file by combining the selected subset of content blocks and the input file. This technique may enable the creation of unique and new musical accompaniments by re-purposing audio or MIDI content from back catalogs and/or out-takes of musical works. The new arrangement may be provided in multiple music styles, genres, or moods and may contain performances from multiple musical instruments, which may be pre-recorded from live instrument performances and/or of MIDI generated musical content.

CROSS-REFERENCE TO RELATED APPLICATIONS, BENEFIT CLAIM

This application claims the benefit as a Continuation of applicationSer. No. 15/922,188, filed Mar. 15, 2018 the entire contents of which ishereby incorporated by reference as if fully set forth herein, under 35U.S.C. § 120. The applicant hereby rescinds any disclaimer of claimscope in the parent application or the prosecution history thereof andadvise the USPTO that the claims in this application may be broader thanany claim in the parent application.

TECHNICAL FIELD

The present invention relates to a method and system for generatingaudio or MIDI (Musical Instrument Digital Interface) files. Morespecifically, the invention relates to a method and system forgenerating an audio or MIDI output file using a harmonic chord map.

BACKGROUND

In the music industry massive back catalogs of musical works previouslyproduced by recording artists, bands or record companies are unused atall or most major record labels. Many songs, although often very wellknown, are largely exhausted and provide little or no new revenue towriters, record labels and publishers.

There is however a high demand for re-purposing some of these well-knownsongs through re-recording new and alternate versions of songs for usagein production industries, such as in films, trailers, gaming,advertising etc. However, such re-purposing presents a significantproblem and barrier to usage as a result of the time and expenseinvolved in re-recording new versions of old songs.

Additionally, many bands and artists have accumulated a very large backcatalog of unused out-takes from previous works over a number of yearsand sometimes decades. Such out-takes are songs or parts of songs thatnever made it to album release. These out-takes are often in audio fileformat, but can include MIDI format also. These files often containparts that can be considered hidden gems that become hard to find orindeed hard to use or reuse.

It is therefore an object of the present invention to provide a systemand method for generating audio or MIDI output files using musicalcontent derived from such back catalogs and/or out-takes of audio orMIDI content.

Further aspects of the present invention will become apparent form theensuing description which is given by way of example only.

SUMMARY

According to an embodiment, there is provided a computer implementedmethod for generating an audio or MIDI output file, the method includingusing one or more processors to perform steps of:

generating audio or MIDI content blocks from one or more musicalperformances;

receiving an input file having audio or MIDI music content;

determining a chord structure and tempo for the input file;

generating a harmonic chord map from the chord structure and tempo forthe input file;

using the harmonic chord map to automatically select content blocks, andgenerating the output file by combining the selected content blocks andthe input file.

The step of generating audio or MIDI content blocks may include stepsof:

receiving an audio or MIDI file including content in a harmonic key froma musical performance;

determining the harmonic key and adapting the musical performance tocorrespond to at least one alternative harmonic key, and

recording said adapted musical performance in said alternative harmonickey to generate the audio or MIDI content blocks.

The step of generating audio or MIDI content blocks may include thefurther step of:

adapting the musical performance to correspond to all harmonic keys,including minor and major keys, and to a plurality of different tempos,

recording said adapted musical performance in all of said harmonic keysand tempos to generate a plurality of audio or MIDI content blocks.

Alternatively, the step of generating audio or MIDI content blocks mayinclude steps of:

receiving an audio or MIDI file including content in a harmonic key fromat least one musical performance;

determining the harmonic key of the musical performance, and

storing each received audio or MIDI file as an audio or MIDI contentblock.

Audio or MIDI content blocks may be stored and retrievable from storageaccording to musical instrument type, tempo and/or harmonic key of themusical performance.

Alternatively, the step of using the harmonic chord map of the inputfile to select content blocks includes steps of:

selecting at least one instrument type;

identifying content blocks for the or each instrument type that matchthe tempo of each bar of music in the harmonic chord map, and

for each selected instrument type and for each bar of music in theharmonic chord map automatically selecting a content block from theidentified content blocks that is in the same harmonic key or that has apart or section that is in the same harmonic key as the bar of music.

The method may include a step of removing sections of a selected contentblock that are not in the same harmonic key as the bar of music of theharmonic chord map.

The harmonic chord map may be generated automatically. Alternatively,the harmonic chord map is generated by a user manually inputting a chordstructure and tempo for the input file.

The output file generated may be in an audio or MIDI file format.

The steps for generating audio or MIDI content blocks may be performedon a plurality of musical performances, in which each musicalperformance includes musical content from a different musicalinstrument.

The step of generating audio or MIDI content blocks may include thefurther step of adapting the musical performances according to one ormore of different: moods, styles and/or genre.

The method may include a step of generating audio or MIDI content blocksfor a plurality of musical performances in all twelve harmonic keys, inboth major and minor keys (i.e. 24 harmonic keys), in a plurality ofdifferent tempos, and according to one or more of different moods,styles and/or genres.

The method may include a step of cutting the audio or MIDI contentblocks to desired intervals/or and lengths.

The method may include storing the audio or MIDI content blocks instorage means.

The step of using the using the harmonic chord map of the input file toselect content blocks may include steps of:

determining the harmonic key and tempo of at least one bar in theharmonic chord map;

selecting a content block corresponding to or matching a musicalinstrument for the at least one bar according to the determined harmonickey and tempo.

In this way, the content blocks are arranged according to the harmonicchord map and used to create the audio or MIDI output file have the sametempo and harmonic map, which ensures the harmonic integrity of theoutput file generated.

The step of using the using the harmonic chord map of the input file toselect content blocks may include steps of:

determining the harmonic key and tempo of each bar in the harmonic chordmap;

selecting content blocks corresponding to a plurality of selectedmusical instruments for each bar according to the determined harmonickey and tempo.

Each bar of the harmonic chord map defines a linear time slot and eachcontent block selected is assigned to a time slot of the harmonic chordmap.

The step of using the using the harmonic chord map of the input file toselect content blocks may be performed automatically or manually by auser.

Content block selection may be further refined according to one or moreof mood, style and/or genre. Such music genre, music style and moodselection is provided as input by a user to user interface means via amenu, such as a drop down menu, an icon based menu or any other suitablemeans which provides users with the capability to view and select from aplurality of options regarding music genre, style and/or mood data orother user preference as required or as desired.

The step of refining a content block according to one or more of mood,style and/or genre may be performed before and/or after generation ofthe output file to provide transitions within the audio or MIDI outputfile between alternative moods styles and/or genres.

The method may include steps of assigning a unique identifier to eachcontent block and using the unique identifiers to select a uniquecombination of content blocks.

The unique identifiers of the selected content blocks are compared withrecords in a storage device, and if a record of a combination of theunique identifiers is located then an output file based on thecombination of selected blocks is not generated. However, if a record ofthe combination of unique identifiers for the selected blocks is notlocated then this indicates the selection is unique and has not beengenerated before, and so the selected content blocks are combined togenerate an audio or MIDI output file.

The method may include the step of storing a record of the combinationof unique identifiers for the selected blocks of the audio of MIDIoutput file generated in a storage device.

In another embodiment of the invention, the method includes the step ofoperating a hold and replace module to hold some of the content blocksof the selected combination and replace others of the selectedcombination with alternative content blocks according to user preferenceinput and matching the harmonic chord map.

The method may include the step of operating a shuffle module toautomatically hold some of the content blocks of the selectedcombination and replace others of the selected combination withalternative content blocks matching the harmonic chord map.

In another embodiment of the invention, the method includes the step ofoperating an audio editing and mixing module to modify, refine, adjust,vary and/or change audio or MIDI characteristics of the output filegenerated.

In another embodiment of the invention, the method includes the step ofoperating an editing and mixing module to modify, refine, adjust, varyand/or change characteristics of a content block.

In another embodiment of the invention, the method includes the step ofoperating an audio editing and mixing module to adjust the tempo and/orshift the harmonic key of a content block. Such editing means and mixingmeans provides a user with full creative control over mix and otherparameters to modify as desired the output file generated and contentblocks used to form the output file.

The method may include the step of importing other instrumentperformances and/or voice from an external source for incorporation intothe output file.

The method may include the step of operating a synchronisation module tosynchronise the output file with an audio or MIDI file, a video file, astill image file, and/or a text file.

In another embodiment of the invention, the method includes the step ofstoring each output file generated in a storage device for retrieval andre-use.

The method may include the step of linking a plurality of storagedevices to provide an extended storage library of content blocks andaudio or MIDI output files for creating further content blocks forre-use.

Embodiments of the invention are operable to receive an audio or MIDIinput file of a song, such as from a back catalog of music (including avocal melody and/or instrument parts), determine a harmonic chord map ofthe song structure and then generate an audio or MIDI output file bycombining pre-recorded content blocks with the input file recording ofthe original song. The content blocks are selected to harmonicallysupport the input file recording of the original song and the process isperformed automatically and in real time.

To generate content blocks for use in providing an accompaniment orbacking track for the original input file the present invention adaptsmusical performances to correspond to all harmonic keys, including minorand major keys, and to a plurality of different tempos and moods. Thecontent blocks may be derived from live instrument performances and/orof MIDI generated musical content. By repeating the process contentblocks may be created for a plurality of different instruments, aplurality of performances for each instrument, in multiple genres, andat different moods and styles in multiple tempos recorded in the twelvekeys (major and minor).

Alternatively, out-takes of audio or MIDI files derived from previousworks may be stored and retrieved as content blocks.

The content blocks may then be automatically selected and whererequired, cut into time slices or slots at required intervals and in therequired harmonic to provide a musical accompaniment for the originalinput file such that the original input file derived vocal melody willbe accompanied by the selected content blocks, even though the musicprovided by the content blocks were not present in the originalrecording of that song, to provide an audio or MIDI output file.

The present invention is adapted to enable bands or artists to write newsongs in their basic format which is inputted as an audio vocal or MIDIfile for the new song. The invention is configured, on receiving aninput file, to search through archived files of such out-takes and tocut relevant usable sections from many different unrelated recordings tocombine relevant audio or MIDI content blocks together for any ormultiple different instruments to thereby create a new backing trackaudio or MIDI file for the new song from previously unused recordedcontent in that band or artists personal content archive.

The present invention makes it possible to successfully generate aunique new musical accompaniment for any input file including an audioor MIDI file of a song (including a vocal melody and/or instrumentparts). The new musical arrangement can be in any, or even multiplemusic styles, genres, or emotional moods and can contain performancesfrom multiple musical instruments, which may be either pre-recorded fromlive instrument performances and/or of MIDI generated musical content.

The audio or MIDI output file generated includes a combination of theselected content blocks and the audio or MIDI input file of an originalsong. The content blocks used in the output file are uniquely tagged toensure the musical arrangement of the selection of content blocks isunique.

Embodiments of the invention enable the entire process also beingachieved entirely through audio and through MIDI format file types, withfinished performances including the audio or MIDI output file beingdelivered as audio or MIDI files so that instrument sounds can bechanged or applied outside of a DAW (Digital-Audio-Workstation) orsimilar third party software applications. It is also possible to importexternally created audio or MIDI files and have those audio or MIDIfiles adapted and expanded to the twelve basic harmonic keys both Majorand Minor. Imported MIDI files are then adapted through autore-composition and exported to match the template map of any chosen songor melody.

In a still further embodiment of the invention, there is provided anon-transitory machine-readable medium including instructions that, whenexecuted by one or more processors, cause the one or processors toperform the steps according to a method as described.

According to embodiments of the invention, there is provided a computerimplemented system for generating an audio or MIDI output file, thesystem including:

means for generating audio or MIDI content blocks from one or moremusical performances;

means for receiving an input file having audio or MIDI music content;

means for determining a chord structure and tempo for the input file;

means for generating a harmonic chord map from the chord structure andtempo for the input file;

means for using the harmonic chord map to automatically select contentblocks, and

means for generating the output file by combining the selected contentblocks and the input file.

The system may further include means for receiving an audio or MIDI fileincluding content in a harmonic key from a musical performance;

means for determining the harmonic key and adapting the musicalperformance to correspond to at least one alternative harmonic key, and

means recording said adapted musical performance in said alternativeharmonic key to generate the audio or MIDI content blocks.

The system may include means for adapting the musical performance tocorrespond to all harmonic keys, including minor and major keys, and toa plurality of different tempos, and means for recording said adaptedmusical performance in all of said harmonic keys and tempos to generatea plurality of audio or MIDI content blocks.

The system may further include means for generating audio or MIDIcontent from a plurality of musical performances, in which each musicalperformance includes musical content from a different musicalinstrument.

The means for generating audio or MIDI content blocks may include meansfor adapting the musical performances according to one or more ofdifferent moods, styles and/or genre.

The system may include means for generating audio or MIDI content blocksfor each musical performance in all twelve harmonic keys, in both majorand minor keys (i.e. 24 harmonic keys), in a plurality of differenttempos, and according to one or more of different moods, styles and/orgenres.

The system may include means for cutting the audio or MIDI contentblocks to desired intervals/or and lengths.

The system may include means for storing the audio or MIDI contentblocks in storage means.

The system may include means for determining the harmonic key and tempoof at least one bar in the harmonic chord map, and means for selecting acontent block corresponding to a musical instrument for the at least onebar according to the determined harmonic key and tempo.

The system may include means for determining the harmonic key and tempoof each bar in the harmonic chord map; and means for selecting a contentblock corresponding to a plurality of selected musical instruments foreach bar according to the determined harmonic key and tempo.

The system may include means for receiving an audio or MIDI fileincluding content in a harmonic key from at least one musicalperformance;

means for determining the harmonic key of the musical performance, andmeans for storing each received audio or MIDI file as an audio or MIDIcontent block.

The system may include means for storing and retrieving audio or MIDIcontent blocks according to musical instrument type, tempo and/orharmonic key of the musical performance.

The system may include means for selecting at least one instrument type;

means for identifying content blocks for the or each instrument typethat match the tempo of each bar of music in the harmonic chord map, and

means for automatically selecting a content block from the identifiedcontent blocks that is in the same harmonic key or that has a part orsection that is in the same harmonic key as the bar of music in theharmonic chord map for each selected instrument type.

The system may include means for removing sections of a selected contentblock that are not in the same harmonic key as the bar of music of theharmonic chord map.

The system may include means for assigning a unique identifier or tag toeach content block and using the unique identifiers to select a uniquecombination of content blocks.

Use of the unique identifiers for the combination as a registrationnumber restricts the combination or anything close from ever recurringthus ensuring the accompaniment file generated is unique and original. Arecord of the combination of unique identifiers for the selected contentblocks in the audio of MIDI output file generated in a storage device.Each output file generated is stored in a storage device for retrievaland re-use and a plurality of storage devices may be linked to providean extended storage library of content blocks and audio or MIDI outputfiles for creating further content blocks for re-use.

The system may include storage means for storing a record of thecombination of unique identifiers for the selected blocks of the audioof MIDI output file generated in a storage device.

The system may include a hold and replace module to hold some of thecontent blocks of the selected combination and replace others of theselected combination with alternative content blocks according to userpreference input and matching the harmonic chord map.

The system may include a shuffle module to automatically hold some ofthe content blocks of the selected combination and replace others of theselected combination with alternative content blocks matching theharmonic chord map.

The system may include an audio editing and mixing module operable tomodify, refine, adjust, vary and/or change audio or MIDI characteristicsof the output file generated.

The editing and mixing module may be further operable to modify, refine,adjust, vary and/or change characteristics of a content block.

The editing and mixing module may be operable to adjust the tempo and/orshift the harmonic key of a content block.

Such editing and mixing means provides a user with full creative controlover mix and other parameters to modify as desired the output filegenerated and content blocks used to form the output file.

The system may include means for importing other instrument performancesand/or voice from an external source for incorporation into the outputfile.

The system may include a synchronisation module to synchronise theoutput file with another audio or MIDI file, a video file, a still imagefile, and/or a text file.

The system may include means for linking a plurality of storage devicesto provide an extended storage library of content blocks and audio orMIDI output files for creating further content blocks for re-use.

There is also provided a computing device and/or arrangement ofcomputing devices having one or more processors, memory and displaymeans operable to display an interactive user interface having thefeatures as described.

In another embodiment of the invention, there is provided a system forgenerating an audio or MIDI output file for adapting an audio or MIDIinput file substantially as herein described with reference to and asshown in the accompanying drawings and/or examples.

In another embodiment of the invention, there is provided a method forgenerating an audio or MIDI output file for adapting an audio or MIDIinput file substantially as herein described with reference to and asshown in the accompanying drawings and/or examples.

DRAWINGS

Embodiments of the invention will be more clearly understood from thefollowing description, given by way of example only, with reference tothe accompanying drawings, in which:

FIGS. 1 and 2 are stylised schematics showing steps in a method forgenerating an audio or MIDI output file according to the presentinvention;

FIG. 3 is a block diagram showing recordings of a piano performance in aset of harmonic keys according to the invention;

FIGS. 4 to 8 are flow diagrams showing an example of the presentinvention in use in generating an audio or MIDI output file;

FIG. 9 is a flow diagram showing steps in the method according to thepresent invention in use in an externally created MIDI file;

FIGS. 10 to 14 are flow diagrams showing a further example of thepresent invention in use in generating an audio or MIDI output file, and

FIG. 15 shows a system for adapting an audio or MIDI input file for usein generating an output file according to the present invention.

DETAILED DESCRIPTION

A method and system according to the present invention is implemented byone or more computer processors and memory including computer softwareprogram instructions executable by the one or more processors. Thecomputer processors may be provided by a computer server or network ofconnected and/or distributed computers.

The audio and/or MIDI input and output files of the present inventionwill be understood to be received, stored or recorded files containingaudio or MIDI data or content which produce sound output when processedby an audio or MIDI player. An audio file may be received or recorded inaudio file formats, including, but not limited to, audio WAV format, MP3format, advanced audio coding (AAC) format, Ogg format or in any otherformat, analog, digital or otherwise, as required. The desired audio orMIDI format may optionally be specified by a user.

A harmonic chord map will be understood to be a chord or harmonicprogression map, which is a representation of the tempo and progressionof successive musical chords or harmonic keys in a musical work.

Referring to the drawings and initially to FIG. 3, the method includesan initial step 10 of generating audio or MIDI content blocks from oneor more musical performances. Step 10 includes receiving an audio orMIDI file including a musical performance, determining the harmonic keyof the performance and adapting the musical performance to correspond toall harmonic keys and at a plurality of different tempos.

As shown in FIG. 3, and by way of example only, a file corresponding toa piano performance, ‘PIANO P1’ 12 has been received and determined asbeing in the C minor harmonic key at a tempo of 100 beats per minute(bpm). The combination of the harmonic key and tempo defines a harmonicchord map for the musical performance file 12.

It will be understood however that a musical performance may havemultiple portions or bars, each sequential bar having the same or adifferent harmonic key and tempo. It will be also understood that thereare twelve (12) basic harmonic keys upon which music may be composed andrecorded. These harmonic keys also have other harmonic factors to beconsidered such as major/minor.

Once the harmonic chord map for the performance ‘PIANO P1’ 12 isdetermined as being in C minor and 100 bpm, it is then recorded in alltwelve minor key sets at 100 bpm, namely:

Cm—C minor, C#m—C sharp minor, Dm—D minor, D#m—D sharp minor, Em—Eminor, E#m—E sharp minor, Fm—F minor, F#m—F sharp minor, Gm—G minor,G#m—G sharp minor, Am—A minor, A#m—A sharp minor and Bm—B minor,and in all 12 major key sets at 100 bpm, namely:CMaj—C major, C#Maj—C sharp major, DMaj—D major, D#Maj—D sharp major, EMaj—E major, E# Maj—E sharp major, F Maj—F major, F# Maj—F sharp major,G Maj—G major, G# Maj—G sharp major, A Maj—A major, A# Maj—A sharp majorand B Maj—B major.

These twenty four piano performance recordings at 100 bpm, indicatedgenerally by the reference numeral 14, are then recorded at a pluralityof different tempos, such as 110 bpm, 115 bpm, 120 bpm, 125 bpm, . . . ,160 bpm and so on, up to a predefined maximum tempo limit.

By creating musical performances, such as a piano performance and thenrecording the same performance in all eleven (12) keys (including themajor and minor) and at multiple tempos, the entire musical spectrum forthat musical performance ‘PIANO P1’ is provided. It is these recordingsfor the entire musical spectrum for a musical performance received thatare used to generate a number of content blocks 16, as shown in FIG. 3.

The steps may be repeated for many different musical performances inwhich each musical performance includes musical content from a differentmusical instrument, such as drums, bass, guitar, strings and otherinstruments as required or as desired. The content blocks 16 may befurther adapted according to one or more of different moods, stylesand/or genres.

The method therefore includes generating audio or MIDI content blocks 16for each of many different musical performances 10 in all twelveharmonic keys, in both major and minor keys (i.e. twenty four harmonickeys), at a plurality of different tempos, and according to onedifferent moods, styles and/or genres and storing the audio or MIDIcontent blocks in storage means.

The audio or MIDI content blocks may optionally be formatted to desiredintervals and lengths as required and a unique identifier is alsoassigned to each content block recorded.

Turning now to FIGS. 1 and 2, the method 1 includes a step 20 ofreceiving an input file (shown as Song A) having audio or MIDI musiccontent. The input file may be a vocal melody and may include both vocaland/or musical performances from back catalogs of pre-recorded musicand/or live music.

At step 22, a chord structure (that is, the harmonic keys making up theperformance) and tempos thereof for the input file is determined and aharmonic chord map is generated. In the instance shown in FIG. 1, theharmonic chord map 24 is determined as including, in a sequential order,the following six harmonic keys or bars at a tempo of 100 bpm: C Major—AMinor—C Major—A Minor—G Major—F Major

At step 26, a content block selection means automatically or by userinput determines an instrument or instruments to be included in theaccompaniment for the input file Song A, and it having been determinedthat the input file requires a piano accompaniment, at step 28,identifies from a storage means specific or donor database specificcontent blocks that match the harmonic chord map 24 for such a pianoaccompaniment.

At step 30, content blocks corresponding to a musical instrument ‘PIANOP1’ and having the determined harmonic key and tempo for the harmonicchord map are selected by the content block selection means. The contentblock selection means selects content blocks from a database or libraryof stored content blocks for a selected instrument, such that eachcontent block selected matches the chord structure and tempo of theinput file ‘Song A’.

The selection of the instrument or instruments to include may beautomatically performed by the content block selection means, and/or maybe specified by a user according to user preferences. Such a selectionis performed by initially analysing all content blocks from relevantkeys and tempos for the specific instrument for a match and thenselecting such matching content blocks for each bar C Major—A Minor—CMajor—A Minor—G Major—F Major of the input file.

At step 32, the selected content blocks are time sliced or cut to thedesired length in order to fit the bars and arranged along the harmonicmap of the input file. Thus each bar of the harmonic chord map defines alinear time slot and each content block selected is assigned to a timeslot of the harmonic chord map.

Thus as shown for an input file with the an original chord structure CMajor—A Minor—C Major—A Minor—G Major—F Major at 100 bpm, the contentblocks selected, for example, for a piano accompaniment, will also havea chord structure C Major—A Minor—C Major—A Minor—G Major—F Major at 100bpm. The relevant and acceptable content blocks for an instrument arearranged along the harmonic chord map in the correct order of harmonicchord marking with a tempo time bar and beat markers.

The process is then repeated thus adapting other selected instrumentcontent blocks as required. For example, the process may select drums,bass, guitar, strings etc and adapt content blocks for these instrumentsfor the harmonic chord map of the original input file. Content blocksmay be further selected or refined according to one or more of mood,style and/or genre, and such selection is provided as an input requestby a user. Such a step of refining a content block may performed beforeand/or after generation of the output file to provide transitions withinthe audio or MIDI output file between alternative moods styles and/orgenres.

The combination and arrangement of selected content blocks along theharmonic map of the input file thus forms an instrument accompaniment orbacking track for the original input file song. The unique identifiersof the selected content blocks are compared with records in a storagedevice to ensure that the combination of content blocks selected isunique.

At step 34, the original input file is aligned to the chord structureharmonic and time map markings to ensure correct placement for playbackwith the instrument accompaniment or backing track including theselected content blocks.

At step 36, an audio or MIDI output file including the selected contentblocks and the original input file song is generated to provide anadapted version of the vocal melody for playback.

The present invention further provides a suite of tools to provide auser with full creative control over mix and other parameters to modifyas desired the audio or MIDI output file generated and content blocksused to form the output file, including:

A hold and replace module to enable users to hold or retain some of thecontent blocks of the selected combination and replace others of theselected combination with alternative content blocks according to userpreference input and matching the harmonic chord map.

A shuffle module to automatically hold some of the content blocks of theselected combination and replace others of the selected combination withalternative content blocks matching the harmonic chord map.

An editing and mixing module to modify, refine, adjust, vary and/orchange audio or MIDI characteristics of content blocks and/or the outputfile generated and to adjust the tempo and/or shift the harmonic key ofa content block.

An importing module to enable live or recorded instrument performancesand/or voice to be imported from an external source for incorporationinto the output file.

A synchronisation module to synchronise the output file with an audio orMIDI file, a video file, a still image file, and/or a text file.

The present invention enables the entire process also being achievedentirely through audio and through MIDI format file types, with finishedperformances including the audio or MIDI output file being delivered asaudio or MIDI files so that instrument sounds can be changed or appliedusing a DAW (Digital-Audio-Workstation) or other software applications.It is also possible to import externally created audio or MIDI files andhave those audio or MIDI files adapted and expanded to 12 basic harmonickeys both Major and Minor. Imported MIDI files are then adapted throughauto re-composition and exported to match the template map of any chosensong or melody.

An example of the present invention for use in adapting an input filewill be described with reference to FIGS. 4 to 9.

As shown in FIG. 4, at step 40, an input file having the title ‘Let itBe’, a well known song by the band The Beatles is received as input. Atstep 42, the harmonic chord map for the song is determined based on thevarious harmonic keys and tempos for each bar of the song. FIG. 6 showsthe harmonic chord map 60 for the song ‘Let it Be’ is determined asincluding bars with harmonic keys C, G and A Major, all at a tempo of110 bpm.

At step 44, this harmonic chord map is provided as input to the contentblock selection means for selection of content blocks to be used ingenerating a musical backing track or accompaniment for the song ‘Let itBe’ according to the harmonic chord map.

As shown in FIG. 5, at step 46, the content block selection meansautomatically or by user driven input determines that the backing trackor accompaniment for the input fie song ‘Let it Be’ should include apiano instrumental part, and so examines all stored piano content blocksP1 and, at step 48, selects content blocks from the Piano P1 contentblocks matching the harmonic map and tempo of the harmonic chord map. Asshown at step 62 of FIG. 6, content blocks from repository ‘Piano P1’that were recorded in the harmonic keys C, G and A Major and at a tempoof 110 bpm are selected thus matching the harmonic chord map of theoriginal input file for ‘Let it Be’. As shown in FIG. 5, step 50, theseselected content blocks are applied and matched to the harmonic chordmap of the original input file song ‘Let it Be’. In this way, acceptablecontent blocks for an instrumental accompaniment that harmonicallysupports the original input file are selected arranged along theharmonic chord map of the input file, in the correct order of harmonicchord marking with a tempo time bar and beat markers.

As shown in FIG. 7, this process is then repeated thus selecting othercontent blocks for different instruments as required. For example, theprocess may additionally select content blocks for a drum accompaniment52, bass 54, guitar 56, strings 58 and select content blocks for theseinstruments for the harmonic chord map of the original input filereceived at 40. As shown, the original input file 40 is aligned to thechord structure harmonic and time map markings to ensure correctplacement for playback with the instrument accompaniment or backingtrack including the selected content blocks 50, 52, 54, 56, 58.

As shown in FIG. 8, at step 64, the unique identifiers of the selectedcontent blocks are compared with records in a storage device to ensurethat the combination of content blocks selected is in the selectedarrangement of content blocks 50, 52, 54, 56, 58 is unique, and at step,66 an audio or MIDI output file including the arrangement of contentblocks 50, 52, 54, 56, 58 and the original input file song 40 isgenerated to provide an adapted unique version of the original inputfile for playback.

FIG. 9 is a flow diagram showing steps in the method according to thepresent invention in an externally created MIDI file. As shown, at step70, the externally created melody is in the musical key of C Major andneeds to be adapted, for example, to the musical key of A minor toharmonically work with the harmonic chord map of a song received as aninput file. At step 72, the external MIDI file is imported or receivedby the system of the present invention where it is adapted. As shown instep 74, the adaptation is achieved by auto-selecting notation which isnot in the musical key of A minor and moving them to the nearest noteposition that is correct for the key of A minor. All other notation thatis in common to both musical keys C Major and A minor are leftunaltered. At step 76, the adapted MIDI file is output.

FIGS. 10 to 14 show a further example of the present invention in use ingenerating an audio or MIDI output file in which the process ofgenerating content blocks differs to that described with reference toFIGS. 4 to 9.

As shown in FIG. 10, the step of generating audio or MIDI content blocksincludes an initial step of receiving audio or MIDI files 78 includingcontent in a harmonic key from at least one musical performance anddetermining the harmonic key of the musical performance. Once determinedeach received audio or MIDI file 78 is stored in storage means 80 as anaudio or MIDI content block. Thus multiple musicians, composers orothers provide unrelated single instrument performance recordings ofvarying lengths and of varying tempos and harmonic structure to acollective data bank for storage in either local or cloud based storageas content blocks.

The audio or MIDI content blocks are stored and retrievable from storagemeans 80 according to musical instrument type, tempo and/or harmonic keyof the musical performance. As shown in FIG. 11, at step 82, an inputfile having audio or MIDI music content is received, such as via a userinterface means. In this way, users, such as singers or songwriters mayinput a song melody and or a measured harmonic chord structure of a songor musical passage. At step 84, the input file or song melody isanalysed and a harmonic chord map and tempo 86 is automaticallygenerated for that song melody. Alternatively a user may manually inputa chord structure for a song.

At FIG. 12, the steps involved in using the harmonic chord map of theinput file 82 to select content blocks are shown. Initially, a userselects from a user interface an instrument type (such as guitar, bass,piano etc) from a selection menu. Such a selection may also be based onmood and/or musical style.

At step 88, content blocks 90 that match the tempo of music in theharmonic chord map are identified. In the instance shown, the tempo wasdetermined for the music represented in the harmonic chord map as 120bpm and so content blocks having a tempo of 120 bpm are identified. Itwill however be understood that tempo changes at different parts of aharmonic chord map will result in identifying content blocks at thosedifferent tempos.

As shown in FIG. 13, at step 92, content blocks 94 at the determinedtempo of 120 bpm for the harmonic chord map and for a selectedinstrument type, in the instance shown, as a guitar, are identified.Then for the selected instrument type and for each bar of music in theharmonic chord map, at step 96, content blocks are selectedautomatically from the identified content blocks that are in the sameharmonic key or that has a part or section that is in the same harmonickey as the bar of music for the instrument.

Thus different parts of different content blocks derived from differentmusical performances for the selected instrument are effectively cut outand assembled along the harmonic chord map of the input file to providean ensemble backing track for the input file. Sections or parts of aselected content block that are not in the same harmonic key as the barof music of the harmonic chord map may be removed to ensure harmonicintegrity with the underlying harmonic chord map.

The present invention thus analyses each content block stem for harmonicchord compatibility. When content blocks are found to have whole orpartial harmonic compatibility, those parts are cut out and applied tothe part or section of the inputted song structure's harmonic chord mapthat is deemed to be a match.

As shown in FIG. 14, at step 97, the process is then repeated for alldesired instruments the user has selected to include in the accompanyingbacking track for the original input song. In the instance shown,content blocks 98 are selected for instrument types, including guitar,bass, piano, banjo, organ and drums, although it will be understood thatany alternative or additional combinations of instruments may be used asrequired or as desired.

As shown in step 99, an audio or MIDI output file including thearrangement of content blocks 98 and the original input file song 82 isgenerated to provide an adapted unique version of the original inputfile for playback.

The present invention is thus further adapted to enable bands or artiststo write new songs in their basic format which is inputted as an audiovocal file or MIDI file for the new song. The invention is configured tosearch through archived files and cuts relevant usable sections frommany different unrelated recordings to combine relevant audio or MIDIcontent blocks together for any or multiple different instrumentsthereby creating a new backing track audio or MIDI file for the new songfrom previously unused recorded content in that band or artistspersonnel content archive.

It is also possible to shuffle an instrument thus replacing it for apreferred performance at which point the system repeats the entireprocess for that chosen instrument.

Furthermore the present platform and architecture can be applied tofurther enhance editing, mixing and shuffling of instruments. It is alsopossible to have a plurality of moods, music styles and genrestransition from one to another within the same song structure and vocalperformance. In other words the song could start as a country song andat any point transition to hip-hop, Latin, rock, orchestral etc allwithin the same track as it plays out. As described using the processabove the present invention will also apply a unique identifier to everyoutput file generated to ensure that every output file is entirelyunique and thus never replicated for the same or any other song or vocalmelody inputted and received.

FIG. 15 shows a system 100 for generating an audio or MIDI output fileto adapt an audio or MIDI input file according to the present invention.The system 100 includes means 110 for generating audio or MIDI contentblocks from one or more musical performances; means 120 for receiving aninput file having audio or MIDI music content; means 130 for determininga chord structure and tempo for the input file; means 140 for generatinga harmonic chord map from the chord structure and tempo for the inputfile; content block selection means 150 for using the harmonic chord mapto automatically select content blocks, and means 160 for generating theoutput file by combining the selected content blocks and the input file.

The content block selection means 150 is further configured fordetermining the harmonic key and tempo of each bar in the harmonic chordmap of the input file and for selecting content blocks corresponding toa plurality of selected musical instruments for each bar according tothe determined harmonic key and tempo.

The content block selection means 150 is also configured for identifyingcontent blocks for the or each instrument type that match the tempo ofeach bar of music in the harmonic chord map, and for automaticallyselecting a content block from the identified content blocks that is inthe same harmonic key or that has a part or section that is in the sameharmonic key as the bar of music in the harmonic chord map for eachselected instrument type.

The system also includes means 170 for receiving an audio or MIDI fileincluding content in an harmonic key from a musical performance for usein generating the content blocks; means 180 for determining the harmonickey and adapting the musical performance to correspond to at least onealternative harmonic key, and means 190 for recording said adaptedmusical performance in said alternative harmonic key to generate theaudio or MIDI content blocks.

The means 190 is further operable for storing each received audio orMIDI file as an audio or MIDI content block and for retrieving audio orMIDI content blocks according to musical instrument type, tempo and/orharmonic key of the musical performance.

The means 190 is further configured for adapting a musical performanceto correspond to all harmonic keys, including minor and major keys, andto a plurality of different tempos, and for recording said adaptedmusical performance in all of said harmonic keys and tempos to generatea plurality of audio or MIDI content blocks. The means 190 is alsoconfigured for generating audio or MIDI content from a plurality ofmusical performances, in which each musical performance includes musicalcontent from a different musical instrument. Means 190 is furtherconfigured for adapting content blocks and/or an output file generatedaccording to one or more of different moods, styles and/or genres.

The means 190 is thus configured for generating audio or MIDI contentblocks for each musical performance in all twelve harmonic keys, in bothmajor and minor keys (i.e. 24 harmonic keys), in a plurality ofdifferent tempos, and according to one or more of different moods,styles and/or genres.

The system includes means 200 for cutting the audio or MIDI contentblocks to desired intervals/or and lengths, and means 210 for storingthe audio or MIDI content blocks in storage means. Also shown is means290 for linking a plurality of storage devices 210 to provide anextended storage library of content blocks and audio or MIDI outputfiles for creating further content blocks for re-use.

The system further includes means 220 for assigning a unique identifieror tag to each content block to enable the content block selection means150 to use the unique identifiers to select a unique combination ofcontent blocks.

The system includes storage means 230 for storing a record of thecombination of unique identifiers for the selected blocks of the audioof MIDI output file generated in a storage device.

The system also includes various tools for use by a user to provide fullcreative control over an output file generated, and for content blocks,including a hold and replace module 240; a shuffle module 250; an audioediting and mixing module 260; a synchronisation module 270 and means280 for importing other instrument performances and/or voice from anexternal source for incorporation into the output file.

Aspects of the present invention have been described by way of exampleonly and it should be appreciated that additions and/or modificationsmay be made thereto without departing from the scope thereof as definedin the appended claims.

What is claimed is:
 1. A computer implemented method for generating anaudio or MIDI output file, the method including using one or moreprocessors to perform steps of: generating audio or MIDI content blocksfrom one or more musical performances; receiving an input file havingaudio or MIDI music content; automatically generating a harmonic chordmap for the input file; using the harmonic chord map to automaticallyselect a subset of the audio or MIDI content blocks, and generating theaudio or MIDI output file by combining the subset of the audio or MIDIcontent blocks and the input file.
 2. The method as claimed in claim 1,in which the step of generating the audio or MIDI content blocksincludes steps of: receiving an audio or MIDI file including content ina particular harmonic key from a musical performance; determining theparticular harmonic key and adapting the musical performance tocorrespond to a plurality of harmonic keys, including minor and majorkeys, and to a plurality of different tempos, and recording said adaptedmusical performance in said plurality of harmonic keys and saidplurality of tempos to generate the audio or MIDI content blocks.
 3. Themethod as claimed in claim 2, in which the step of using the harmonicchord map to select the subset of the audio or MIDI content blocksincludes steps of: for each bar in the harmonic chord map, determining aharmonic key and tempo; selecting content blocks corresponding to one ormore musical instruments for each bar in the harmonic chord mapaccording to the harmonic key and tempo for said each bar.
 4. The methodas claimed in claim 1, in which the step of generating the audio or MIDIcontent blocks includes steps of: receiving an audio or MIDI fileincluding content in a harmonic key from at least one musicalperformance; determining the harmonic key of the musical performance,and storing the audio or MIDI file as an audio or MIDI content block. 5.The method as claimed in claim 4, in which the step of using theharmonic chord map to select the subset of the audio or MIDI contentblocks includes steps of: selecting one or more instrument types; foreach instrument type of the one or more instrument types: identifying aset of content blocks for said each instrument type that match the tempoof each bar of music in the harmonic chord map, and for each bar ofmusic in the harmonic chord map, automatically selecting, from the setof content blocks, a content block that is in the same harmonic key, orthat has a part or section that is in the same harmonic key, as the barof music.
 6. The method as claimed in claim 5, further including a stepof removing sections of the content block that are not in the sameharmonic key as the bar of music of the harmonic chord map.
 7. Themethod as claimed in claim 1, including a step of cutting the audio orMIDI content blocks to desired intervals and/or lengths.
 8. The methodas claimed in claim 1, in which the harmonic chord map is generatedautomatically or by a user manually inputting a chord structure andtempo for the input file.
 9. The method as claimed in claim 1, in whicheach bar of the harmonic chord map defines a linear time slot for amusical chord and each content block in the subset is assigned to a timeslot of the harmonic chord map.
 10. The method as claimed in claim 1,wherein selecting the subset is further refined according to one or moreof mood, style, or genre, wherein refining a content block according toone or more of mood, style or genre is performed before or aftergeneration of the audio or MIDI output file.
 11. The method as claimedin claim 1, including a step of assigning a unique identifier to each ofthe audio or MIDI content blocks and using the unique identifiers toselect a unique combination of the audio or MIDI content blocks.
 12. Themethod as claimed in claim 11, including a step of storing a record ofthe unique combination of unique identifiers for the subset of the audioor MIDI content blocks of the audio or MIDI output file generated in astorage device.
 13. The method as claimed in claim 1, including a stepof operating a hold and replace module to hold some of the subset of theaudio or MIDI content blocks and replace others of the subset of theaudio or MIDI content blocks with alternative audio or MIDI contentblocks according to user preference input and matching the harmonicchord map.
 14. The method as claimed in claim 1, including a step ofoperating a shuffle module to automatically hold some of the subset ofthe audio or MIDI content blocks and replace others of the subset of theaudio or MIDI content blocks with alternative audio or MIDI contentblocks matching the harmonic chord map.
 15. The method as claimed inclaim 1, including a step of operating an audio editing and mixingmodule to modify, refine, adjust, vary or change audio or MIDIcharacteristics of a content block or the audio or MIDI output file. 16.The method as claimed in claim 1, including a step of importing otherinstrument performances or voice from an external source forincorporation into the audio or MIDI output file.
 17. The method asclaimed in claim 1, including a step of operating a synchronisationmodule to synchronise the audio or MIDI output file with an audio orMIDI file, a video file, a still image file, or a text file.
 18. Themethod as claimed in claim 1, including a step of storing each outputfile generated in a storage device for retrieval and re-use and linkinga plurality of storage devices to provide an extended storage library ofcontent blocks and audio or MIDI output files for creating furthercontent blocks for re-use.
 19. A non-transitory machine-readable mediumincluding instructions that, when executed by one or more processors,cause the one or processors to perform the steps according to claim 1.20. A computer implemented system for generating an audio or MIDI outputfile, the system including: means for generating audio or MIDI contentblocks from one or more musical performances; means for receiving aninput file having audio or MIDI music content; means for automaticallygenerating a harmonic chord map for the input file; means for using theharmonic chord map to automatically select a subset of the audio or MIDIcontent blocks, and means for generating the audio or MIDI output fileby combining the subset of the audio or MIDI content blocks and theinput file.